Irregular input identification

ABSTRACT

Techniques for identifying irregular objects in contact with, or in close proximity to, a touch-surface are described. An irregularity measure is determined based on the regions intrinsic characteristics (e.g., energy content) rather than on the shape (or pattern) of the pixels within the region.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/584,725, filed Aug. 13, 2012 and published Dec. 6, 2012 as U.S.Patent Publication No. 2012-0306800; which is a continuation of U.S.patent application Ser. No. 11/619,490, filed Jan. 3, 2007 and issuedSep. 18, 2012 as U.S. Pat. No. 8,269,727; the entire disclosures ofwhich are incorporated herein by reference in their entirety for allintended purposes.

BACKGROUND

The invention relates generally to data input methods and devices forelectronic equipment and, more particularly, to methods and devices fordiscriminating between various inputs to a multi-touch touch-surfaceinput device.

There currently exist many types of input devices for performingoperations with an electronic system. These operations often correspondto moving a cursor and/or making selections on a display screen.Illustrative electronic systems include tablet, notebook, desktop andserver computer systems, personal digital assistants, audio and videocontrol systems, portable music and video players and mobile andsatellite telephones. The use of touch pad and touch screen systems(collectively “touch-surfaces”) has become increasingly popular in thesetypes of electronic systems because of their ease of use and versatilityof operation.

One particular type of touch-surface is the touch screen. Touch screenstypically include a touch panel, a controller and a software driver. Thetouch panel is characteristically an optically clear panel with a touchsensitive surface that is positioned in front of a display screen sothat the touch sensitive surface is coextensive with a specified portionof the display screen's viewable area (most often, the entire displayarea). The touch panel registers touch events and sends signalsindicative of these events to the controller. The controller processesthese signals and sends the resulting data to the software driver. Thesoftware driver, in turn, translates the resulting data into eventsrecognizable by the electronic system (e.g., finger movements andselections).

Unlike earlier input devices, touch-surfaces now becoming available arecapable of simultaneously detecting multiple objects as they approachand/or contact the touch-surface, and detecting object shapes in muchmore detail. To take advantage of this capability, it is necessary tomeasure, identify and distinguish between the many kinds of objects thatmay approach or contact such touch-surfaces simultaneously. Prior arttouch-surface systems (including their supporting software and/orcircuitry) do not provide a robust ability to do this. Thus, it would bebeneficial to provide methods and devices that identify and discriminatemultiple simultaneous hover or touch events such as, for example, two ormore closely grouped fingers, palm heels from one or more fingers,fingers from thumbs, and fingers from ears and cheeks.

SUMMARY

In one embodiment the invention provides a method to process a proximityimage from a multi-touch touch surface. The method includes obtaining adispersion image having a plurality of pixels, determining anirregularity measure value for the dispersion image and controlling theoperation of a multi-touch touch-surface device if the irregularitymeasure value is above a specified threshold. In one embodiment, theirregularity measure is determined for the entire proximity image. Inanother embodiment, an irregularity measure is determined for one ormore regions within the image (e.g., patches). Device control may bemanifested through, for example, changing the device's operating mode(e.g., off to on) or ignoring the identified irregular object so that itdoes not cause a change in the device's operating state.

Illustrative multi-touch touch-surface devices include, but are notlimited to, tablet computer systems, notebook computer systems, portablemusic and video systems and mobile telephones. Methods in accordancewith any of the described methodologies may be stored in any media thatis readable and executable by a programmable control device such as, forexample, by a general purpose computer processor.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows, in flowchart form, a multi-touch processing methodology inaccordance with one embodiment of the invention.

FIG. 2 shows, in flowchart form, a patch irregularity calculation inaccordance with one embodiment of the invention.

FIG. 3 shows a plot of empirically determined data illustrating patchminor radii's ability to discriminate between large touch-surfacecontacts (cheeks, for example) and other touch-surface contacts(fingertips and thumbs, for example).

FIG. 4 shows a plot of empirically determined data illustrating patchminor radii's ability to discriminate between palm contacts and othertouch-surface contacts (e.g., fingertips and thumbs).

FIG. 5 shows a plot of empirically determined data illustrating a patchirregularity measure's ability to discriminate between ear contacts andother touch-surface contacts (e.g., fingertips, thumbs and cheeks).

FIG. 6 shows, in flowchart form, far-field operations in accordance withone embodiment of the invention.

FIG. 7 shows, in block diagram form, a touch-surface device inaccordance with one embodiment of the invention.

DETAILED DESCRIPTION

Methods and devices to detect and discriminate between multiplesimultaneous close approaches or touches to a touch-surface aredescribed. The following embodiments are presented to enable any personskilled in the art to make and use the invention as claimed and areprovided in the context of mutual capacitance touch-surface devices.Variations using other types of touch-surfaces such as force or opticalsensing touch-surfaces will be readily apparent to those skilled in theart. Accordingly, the claims appended hereto are not intended to belimited by the disclosed embodiments, but are to be accorded theirwidest scope consistent with the principles and features disclosedherein.

As previously noted, recent touch-surface input devices are capable ofsimultaneously detecting multiple objects as they approach and/orcontact the touch-surface. For a hand-held multi-touch touch-surfacedevice that may be put into a pocket, purse, or held against the head(e.g., portable music player, portable video player, personal digitalassistant or mobile phone), detecting when the device is being claspedon the way into or out of the pocket, against the body, or against thehead is very useful for: input rejection (ensuring that touch-surfaceinput signals generated as a result of these actions are not mistakenfor normal finger/stylus touches); operational mode transitions (e.g.,dimming the device's backlight, putting the device to sleep and wakingthe device from a low-power state); and, for mobile telephones,answering calls (e.g., when the device is brought near, but notnecessarily touching the head) and/or terminating calls (e.g., when theunit is placed into a pocket or purse).

Each sensing element (aka “pixel”) in a two dimensional array of sensingelements (i.e., a touch-surface) generates an output signal indicativeof the electric field disturbance (for capacitance sensors), force (forpressure sensors) or optical coupling (for optical sensors) at thesensor element. The ensemble of pixel values represents a “proximityimage.” As described herein, various embodiments of the inventionaddress the ability to detect and discriminate between touch-surfacesignals (represented as a proximity image) resulting from, for example,the types of actions identified in the previous paragraph.

Referring to FIG. 1, multi-touch processing methodology 100 inaccordance with one embodiment of the invention begins with theacquisition of proximity image data (block 105). Because the acquireddata is usually a superposition of information (indicating an objectclose to or in contact with the touch-surface) fixed offsets (due tocircuitry baselines) and noise (e.g., radio frequency interference), aninitial adjustment to acquired pixel data may be made to compensate forsensor element baseline activity. For example, on multi-touch deviceinitialization and/or when being brought out of a low-power mode (e.g.,sleep), one or more images may be captured. By assuming these initialimages include no surface contacts, they may be used to provide thesensor's baseline. Averaging over multiple sequential images (using, forexample, infinite or finite impulse response filters) has been found toprovide more accurate baseline values. These baseline values may besubtracted from each subsequently captured image to provide a proximityimage for use in ongoing image processing steps. In another embodiment,baseline pixel values may be slowly adjusted over time to compensate fortemperature or static charge. In addition, the initial baseline valuesmay need to be adjusted if, in fact, touch-surface contacts were presentat start-up. In yet another embodiment, a plurality of image samples maybe acquired each at a different sensor element driving frequency. Foreach pixel in these images, the mean or median of subtracted samples(i.e., between the captured baseline and information images) may becombined to create an initial (typically signed) image in accordancewith block 105. For noise that occasionally generates large outlierpixel values (“spiky” noise), other rank-order filters may be useful. Asnoted in FIG. 1, proximity image data resulting from operations inaccordance with block 105 is denoted [PROX].

Next, [PROX] image data feeds other processing blocks that may operatesequentially or in parallel with one another (blocks 110, 115 and 120).It has been found that filtering or smoothing a proximity image (block115) prior to segmentation (block 125) reduces the number of spuriouspeaks and thus helps reduce over segmentation. In one embodiment ofblock 115, each pixel value may be averaged with its nearest neighborpixels in accordance with a discrete diffusion operation. If thisapproach is employed, it has been found beneficial to insert a “border”around the captured image so that there is a value with which to averagethe pixels at the edge of the captured image. For example, a one (1)pixel border may be added to the [PROX] image—where each “border” pixelis assigned a value corresponding to the image's “background” (e.g.,zero). In another embodiment, both temporal (e.g., obtaining multipleimages over a period of time) and spatial (e.g., averaging neighborpixels) smoothing operations may be used. Multiple smoothing operationsmay be beneficial if the captured pixel data is particularly noisy. Asnoted in FIG. 1, image data resulting from operations in accordance withblock 115 is denoted [SMTH].

While [PROX] image pixel values are typically zero or positive inresponse to an object contacting the touch-surface (aka, a “grounded”object), background noise or objects close to but not touching thetouch-surface (aka “ungrounded” objects) may produce an image some ofwhose pixel values are negative. Background noise may be static or varywith circuit temperature, touch-surface moisture, or other factors.Noisy, negative pixels can cause excessive jitter in centroid and otherpatch measurements (see discussion below regarding block [135]). Tocompensate for this, [PROX] image pixel values may be confined to adesired, typically positive, range (block 110). Subtracting the noisethreshold helps reduce centroid jitter induced from pixels that wanderaround (above and below) the noise threshold in successive image frames.As noted in FIG. 1, image data resulting from operations in accordancewith block 110 is denoted [CNST]. In one embodiment, all pixel whosevalues are less than a background noise threshold are set to zero. Inanother embodiment, a noise-threshold is subtracted from each pixelvalue and the result is forced to be non-negative, as shown in Table 1.

TABLE 1 Illustrative Pixel Constraint Technique On a pixel-by-pixelbasis: If [PROX] < (Noise Threshold)    [CNST] = (Background Value) Else   [CNST] = [PROX] − (Noise Threshold)

In one embodiment, the noise-threshold value is set to between 1 and 3standard deviations of the noise measured at each pixel and thebackground-value is set to zero. One skilled in the art will recognizethat other values are possible and that the precise choice of valuesdepends, inter alia, on the type of sensor element used, the actual orexpected level of pixel noise and the multi-touch device's operationalenvironment. For example, the noise threshold may be set to a specifiedexpected value on a per-pixel basis or a single value may be used forall pixels in an image. In addition, pixel noise values may be allowedto vary over time such that thermal and environmental effects on sensorelement noise may be compensated for.

Touch-surface contacts typically show up as grouped collections of“active” pixel values, where each region of fleshy contact (e.g. finger,palm, cheek, ear or thigh) is represented by a roughly elliptical patchof pixels.

By analyzing an image's topography, image segmentation operations canidentify distinct pixel patches that correspond to touch-surfacecontacts (block 125). In one embodiment, bottom-up, ridge-hikingalgorithms may be used to group pixels that are part of the samewatershed around each peak pixel—each watershed group or pixel patchcorresponds to a touch-surface contact. In another embodiment, top-downsearch algorithms may be used to identify pixel patches surrounding eachpeak pixel, starting from the peak, searching outward and stopping atvalleys. As part of the image segmentation process, one-dimensionalpatches may be culled from the identified patches in that they generallyresult from isolated noise spikes or failure of an entire row or columnof sensor elements and/or associated circuitry. In addition, becauselarge contacts such as palms and elongated thumbs may produce multiplepeaks in a proximity image (due to noise or non-uniform signalsaturation, for example), multiple peaks in the image can grow intomultiple, split patches. To account for this phenomenon, multipledetected patches may be merged to produce a reduced number of patchesfor further processing. Heuristic or empirically determined rules may,for example, be applied to accomplish this. For example, two separatelyidentified patches may be merged when the saddle point along theirshared border is not “very deep”—e.g., when the saddle magnitude is morethan 60% to 80% of the two patches' peak pixel values. As noted in FIG.1, identified patches resulting from operations in accordance with block125 are denoted [P1, P2, . . . Pn].

Analysis shows that noise from pixels on the periphery of a patch, farfrom the center or peak pixel, can cause more jitter in calculatedcentroid (center-of-‘mass’) measurements than the same amount of noisefrom central pixels. This phenomenon applies to otherstatistically-fitted patch parameters such as major/minor radii andorientation as well. This jitter can be a particularly serious problemfor the smooth tracking of hovering objects because hovering objects donot generally induce strong central pixels, leaving the peripheralpixels with even greater influence on the centroid measurement. However,completely leaving these peripheral pixels out of a patches' centroidcalculations would discard potentially useful information about theposition, size, and shape of the patch. It is further noted thatperforming patch parameterization on diffused images may reduce noisefrom peripheral pixels, but standard spatial filtering processes alsocause swelling and distortion of patch shape, cause adjacent patches tospread into one another and other effects that bias centroid and ellipseradii measurements in particular. Thus, a technique is needed thatminimizes the amount of noise from patch periphery pixels withoutstrongly distorting patch shape and ensuing measurements.

In accordance with one embodiment of the invention, therefore, patchperipheral pixel values may be selectively reduced, down-scaled ordampened (block 130). Generally, patch centroid determination may beimproved by selectively down-scaling patch peripheral pixels that arefairly weak and whose neighbors are very weak. More specifically, in oneembodiment calibrated image pixel values (e.g., in [CNST]) whosecorresponding smoothed value (e.g., in [SMTH]) falls within a specifiedrange defined by a lower-limit and an upper-limit are reduced inproportion to where the smoothed value falls within that range. Lowerand upper limits are chosen empirically so that only those pixel valuesthat are relatively weak (compared to patch peak values and backgroundnoise) are manipulated. It has been found that: if the lower-limit isset too low, the patch will “bloom” from background pixels that happento have positive noise; if the lower-limit is set too high, the patches'centroid position will have a spatially periodic bias toward sensorelement centers (e.g., capacitive electrode plate centers); if theupper-limit is not sufficiently higher than the lower-limit, peripherydampening will not provide any significant centroid jitter reductionbenefits; and if the upper-limit is too high, all patch pixels besidesthe patches' peak pixel will be affected, again biasing determination ofthe patches' centroid toward sensor element centers. In accordance withone embodiment of the invention, the lower-limit is set, on apixel-by-pixel basis, to approximately twice the background noisestandard deviation and the upper-limit is set to approximately fourtimes the background noise standard deviation (with the background valuetypically being zero). In another embodiment, the lower-limit is set toa value indicative of the “average” or “expected” noise across allpixels in the proximity image. In some embodiments, the noise value maychange dynamically to reflect changing operational conditions (seecomments above). As noted in FIG. 1, an image whose peripheral patchpixels have been dampened in accordance with block 130 is denoted[CNST′]. In one embodiment, peripheral patch pixels are dampened asshown in Table 2.

TABLE 2 Illustrative Peripheral Patch Pixel Dampening   For each pixelin a patch: If [SMTH] < (Lower Limit)  [CNST′] = (Background Value) ElseIf [SMTH] > (Upper Limit)  [CNST′] = [CNST] Else$\lbrack {CNST}^{\prime} \rbrack = {\frac{\lbrack{SMTH}\rbrack - {{Lower}\mspace{14mu}{Limit}}}{{{Upper}\mspace{14mu}{Limit}} - {{Lower}\mspace{14mu}{Limit}}} \times \lbrack{CNST}\rbrack}$

Patch peripheral pixel dampening such as described above is equallyapplicable to touch-surfaces that provide one-dimensional proximityimages. For example, projection scan touch-surfaces provide an outputvalue (or signal) for each row and column of sensor elements in atouch-surface. In these types of touch-surfaces, a “patch” comprises aplurality of values, where each value represents a row or columnmeasurement. The values at the ends of these patches (i.e., theperipheral values) may benefit from noise dampening as described here.

For certain touch-surface input devices such as a telephone, the ear andearlobe may contact the touch-surface sooner or more often than thecheek during calls. Unfortunately, earlobe patches can be very close insize to finger and thumb patches—but should, nevertheless, not causespurious finger-button activations during a call. In accordance with oneembodiment of the invention, a measurement of patch irregularity isdefined that does not look for any specific ear (patch) shape, butrather indicates a general roughness, non-roundness or folds in thepixel patch (block 120). That is, if a patches' irregularity measure isabove a specified threshold, the contact is identified as an irregularobject (e.g., not a cheek, finger or palm), otherwise the patch isidentified as not an irregular object (e.g., a cheek, finger or palm).

Referring to FIG. 2, patch irregularity determination methodology 120begins with the computation of a dispersion image (block 200). Ingeneral, the dispersion image (denoted [DISP] in FIG. 2) may be anyhigh-pass filtered version of the initial proximity image [PROX]. In oneembodiment, the [DISP] image is generated using a form of unsharpmasking as follows:[DISP]=[PROX]−[SMTH]  EQ. 1

Next, the total energy for each patch [P1, P2, . . . Pn] is computed(block 205). In one embodiment, for example, a patches' total energy maybe calculated by summing the square of each pixel value in the patch.This may be expressed mathematically as follows:

$\begin{matrix}{{{Total}\mspace{14mu}{Energy}\mspace{14mu}{in}\mspace{14mu}{Patch}\mspace{14mu} p} = {{Ep} = {\sum\limits_{i,{j\mspace{14mu}{in}\mspace{14mu} p}}\lbrack {DISP}_{{\lbrack i\rbrack}{\lbrack j\rbrack}}^{2} \rbrack}}} & {{EQ}.\mspace{14mu} 2}\end{matrix}$

As noted in FIG. 2, total patch energy values resulting from operationsin accordance with block 205 are denoted [E1, . . . En].

The total energy between adjacent pixels in a patch is then determined(block 210). To reduce the effect of energy spikes for pixel patchesstraddling an edge, the summations below should neglect (i.e., assume avalue of zero) contributions from pixels whose neighboring pixels are atthe image's border, see EQ. 3. For the same reason, the summations belowshould ignore contributions from pixels whose neighboring pixels arefrom a different patch.

$\begin{matrix}{{{Total}\mspace{14mu}{Spatial}\mspace{14mu}{Energy}\mspace{14mu}{for}\mspace{14mu}{Patch}\mspace{14mu} p} = {{SEp} = {\begin{pmatrix}{{\sum\limits_{i,{j\mspace{14mu}{in}\mspace{14mu} p}}( {\lbrack {DISP}_{{\lbrack i\rbrack}{\lbrack j\rbrack}} \rbrack - \lbrack {DISP}_{{\lbrack{i + 1}\rbrack}{\lbrack j\rbrack}} \rbrack} )^{2}} +} \\{{\sum\limits_{i,{j\mspace{14mu}{in}\mspace{14mu} p}}( {\lbrack {DISP}_{{\lbrack i\rbrack}{\lbrack j\rbrack}} \rbrack - \lbrack {DISP}_{{\lbrack{i - 1}\rbrack}{\lbrack j\rbrack}} \rbrack} )^{2}} +} \\{{\sum\limits_{i,{j\mspace{14mu}{in}\mspace{14mu} p}}( {\lbrack {DISP}_{{\lbrack i\rbrack}{\lbrack j\rbrack}} \rbrack - \lbrack {DISP}_{{\lbrack i\rbrack}{\lbrack{j + 1}\rbrack}} \rbrack} )^{2}} +} \\{\sum\limits_{i,{j\mspace{14mu}{in}\mspace{14mu} p}}( {\lbrack {DISP}_{{\lbrack i\rbrack}{\lbrack j\rbrack}} \rbrack - \lbrack {DISP}_{{\lbrack i\rbrack}{\lbrack{j - 1}\rbrack}} \rbrack} )^{2}}\end{pmatrix} \div 4}}} & {{EQ}.\mspace{14mu} 3}\end{matrix}$

The sum is divided by 4 because each pixel gets counted once for eachdirection in the proximity image (left, right, up and down). As noted inFIG. 2, total patch spatial energy values resulting from operations inaccordance with block 210 are denoted [SE1, . . . SEn]. Next, the energyassociated with each patches' peak pixel is determined (block 215) asfollows:

$\begin{matrix}{{{Peak}\mspace{14mu}{Energy}\mspace{14mu}{for}\mspace{14mu}{Patch}\mspace{14mu} p} = {{PEp} = {\max\limits_{i,{j\mspace{14mu}{in}\mspace{14mu} p}}( \lbrack{DISP}\rbrack )^{2}}}} & {{EQ}.\mspace{14mu} 4}\end{matrix}$

As noted in FIG. 2, peak patch energy values resulting from operationsin accordance with block 215 are denoted [PE1, . . . PEn].

Finally, an irregularity measure for each patch is calculated (block220). In one embodiment, the irregularity measure is defined as theratio of a patches' spatial energy minus its peak energy to the patches'total energy:

$\begin{matrix}{{{Irregularity}\mspace{14mu}{Measure}\mspace{14mu}{for}\mspace{14mu}{Patch}\mspace{14mu} p} = {{IMp} = \frac{{SEp} - {PEp}}{Ep}}} & {{EQ}.\mspace{14mu} 5}\end{matrix}$

As noted in FIG. 2, patch irregularity measure values resulting fromoperations in accordance with block 220 are denoted [IM1, . . . IMn].Alternative embodiments might discard any negative pixels valuesgenerated during computation of pixel energy values, or take absolutepixel values rather than squaring pixel values when calculating patchenergies.

In another embodiment, the irregularity measure may be based on theproximity image as a whole. That is, the entirety of the dispersionimage (i.e., all pixels) may be treated as a single “patch” for purposesof generating an irregularity measure value. One benefit to thisapproach is that abnormal touch-surface surface conditions may bedetected, and responded to, prior to segmentation operations inaccordance with block 125 (see FIG. 1). Illustrative abnormaltouch-surface surface conditions include, but are not limited to, liquid(e.g., water or sweat) on the touch-surface or multiple irregularobjects in close proximity to or in contact with the touch-surface(e.g., coins and/or keys). When these conditions are detected, it may bebeneficial to acquire new sensor element baseline values. In addition,if multiple touch-surface sensor sampling frequencies are employed anirregularity measure may be computed at each of the frequencies. If oneor more of the computed irregularity measure values is greater than aspecified threshold as discussed above, the sampling frequenciesassociated with the above-threshold values may be deemed to be affectedby an excessive amount of noise and ignored (e.g., radio frequencynoise). Periodic determination of frequency-dependent irregularitymeasures in this manner may be used to detect when such noise sourcesoccur and when they disappear. For example, due to a touch-surfacedevices operating environment changes.

In general, an oddly shaped collection of pixels (i.e., a patch) canrequire a relatively large set of numbers to define its boundary andsignal value at each pixel within the patch. To reduce the computationalcomplexity of identifying, distinguishing and tracking touch events,however, it is advantageous to characterize patches identified inaccordance with block 125 with as few numbers as practical. Because mostpatches from flesh contact tend to have an elliptical shape, oneapproach to patch parameterization is to fit an ellipse to each patch.One benefit of this approach is that an ellipse is completely describedby a relatively small collection of numbers—its center coordinates,major and minor axis lengths, and major axis orientation.

Referring again to FIG. 1, using this approach known centroid orcenter-of-mass computations may be used to parameterize each patch(block 135). In general, a patches' centroid may be determined usingthese techniques and the [CNST′] image (see block 130). In addition, the[CNST′] image may be used to generate patch covariance matrices whoseEigenvalues identify a patches' major and minor radii and whoseEigenvectors identify the patches' orientation. For contactdiscrimination operations (see discussion below regarding block 140),the following patch characteristics are also computed:

$\begin{matrix}{\mspace{79mu}{{{Total}\mspace{14mu}{Signal}\mspace{14mu}{for}\mspace{14mu}{Patch}\mspace{14mu} p} = {\sum\limits_{i,{j\mspace{14mu}{in}\mspace{14mu} p}}\lbrack {CNST}_{{\lbrack i\rbrack}{\lbrack j\rbrack}}^{\prime} \rbrack}}} & {{EQ}.\mspace{14mu} 6} \\{{{Signal}\mspace{14mu}{Density}\mspace{14mu}{for}\mspace{14mu}{Patch}\mspace{14mu} p} = \frac{( {{Total}\mspace{14mu}{Signal}\mspace{14mu}{for}\mspace{14mu}{Patch}\mspace{14mu} p} )}{( {{Geometric}\mspace{14mu}{Mean}\mspace{14mu}{Radius}\mspace{14mu}{of}\mspace{14mu}{Patch}\mspace{14mu} p} )}} & {{EQ}.\mspace{14mu} 7}\end{matrix}$In another embodiment, patch signal density may be approximated by:

$\begin{matrix}{{{Signal}\mspace{14mu}{Density}\mspace{14mu}{for}\mspace{14mu}{Patch}\mspace{14mu} p} = \frac{( {{Total}\mspace{14mu}{Signal}\mspace{14mu}{for}\mspace{14mu}{Patch}\mspace{14mu} p} )}{( {{Number}\mspace{14mu}{of}\mspace{14mu}{Pixels}\mspace{14mu}{in}\mspace{14mu}{Patch}\mspace{14mu} p} )}} & {{EQ}.\mspace{14mu} 8}\end{matrix}$

Prior art techniques to discriminate between objects that actuallycontact a touch-surface from those that are merely hovering above ithave relied upon a patches' total signal parameter (see, for example,EQ. 6). This approach, however, is very dependent upon the size of theobject being identified. That is, prior art techniques that threshold ona patches' total signal value generally only work well for objects of asingle size. For example, a total patch signal threshold selected toidentify a fingertip contact would trigger detection of a thumb or palmwhen those objects are far above the touch-surface. Such a situationleads to the misidentification of patches (e.g., identifying a patchactually caused by a palm as a thumb).

In contrast, a discrimination technique in accordance with oneembodiment of the invention uses a patches' signal density parameter(see, for example, EQS. 7 and 8). It has been found that this approachprovides a robust means to distinguish objects that contact thetouch-surface from those that are held or hovering above thesurface—regardless of the object's size.

If the patch signal density parameter is normalized such that a firmfingertip contacting the touch-surface produces a peak value of 1, thena lightly brushing contact is indicated by a patch density value ofslightly greater than 0.5 (e.g., half the normalized value) while ahovering object would be indicated by a patch density value equal to orslightly less than 0.5. It will be recognized that what constitutes“slightly greater” or “slightly less” is dependent upon factors such asthe type of sensor elements used in the touch-surface and the associatedsensor element drive and measure circuitry. Accordingly, while theprecise determination of a threshold value based on patch signal densitywill require some experimentation, it would be will within the purviewof an artisan of ordinary skill with benefit of this disclosure.

It has also been determined that fingernail touches also produce patchsignal density values generally greater than approximately 0.5. This isbecause the non-conductive fingernail holds the conductive finger fleshmore than approximately 1 millimeter above the touch-surface.Accordingly, a threshold operation based on patch signal density is alsoa reliable means for discriminating between fleshy fingertip touches andback-of-fingernail touches. This same technique has also been found toreliably determine whether large objects (e.g., cheeks and palms) arehovering or actually in contact with the touch-surface.

With patch parameterization complete, the various types of touch-surfacecontacts may be discriminated (block 140). Using the parametersidentified above, it is possible to robustly and reliably distinguishlarge objects (e.g., cheeks and palms) form other objects (e.g., fingersand thumbs), irregular objects (e.g., ears) from regular objects (e.g.,fingers, thumbs, cheeks and palms) and finger-clasp actions (e.g., whena user claps a multi-touch touch-surface device to put it into orwithdraw it from a pocket). Identification of and discrimination betweenthese types of touch-surface inputs permits an associated device to becontrolled in a more robust manner. For example, in one embodimentdetection of a large object may be used to transition the device fromone operational state (e.g., off) to another (e.g., on). In anotherembodiment, input identified as the result of a large or irregularobject, which might normally cause a state transition, may be safelyignored if in one or more specified states. For example, if atouch-surface telephone is already in an “on” or “active” state,identification of a large or irregular object may be ignored.

As previously noted, it can be advantageous to distinguish large objects(e.g., cheeks and palms) from small objects (e.g., fingertips),regardless of whether the objects are hovering a few millimeters abovethe touch-surface or are pressed firmly against the surface. It has beenfound that a contact's minor radius measure provides a robustdiscriminative measure to accomplish this. If a patches' minor radiusexceeds a specified threshold, the contact can reliably be classified asa cheek—as opposed to a finger or thumb, for example. This samemeasurement can also detect a nearby leg (e.g., thigh) through a fewmillimeters of fabric (e.g. when a device is inserted in the pocket withits touch-surface facing the body). This measurement has been found tobe so robust that if other patches appear on the surface with smallerminor radii (e.g. from an earlobe), they may be safely ignored.Referring to FIG. 3, illustrative empirical data is shown thatillustrates the distinction between cheek contacts 300 and othercontacts 305 (e.g., fingertips and thumbs) based on patch minor radii.While the exact values for patch contacts may vary from sensor to sensorand population to population, it is clear from FIG. 3 that threshold 310may be made anywhere between approximately 11 millimeters andapproximately 15 millimeters. (In this and the following data plots,patch signal density values are normalized to 1 for a fully contactingfingertip.) While threshold 310 is described by a constant value (i.e.,dependent only upon patch minor radius), this is not necessary. Forexample, threshold 310 may be described by a linear or non-linearrelationship between multiple parameters such patch minor-radius andpatch signal density (see discussion below regarding FIG. 4).

A similar size testing may be performed using a patches' major orgeometric mean radius (i.e., √{square root over ((patch major axisradius)(patch minor axis radius))}), the minor-radius discriminationdescribed here has been found to be superior because it is better ableto discriminate between thumbs or flattened fingers. (Flattened fingersmay produce major radii as large as a cheek major radius, but theirminor radii are typically no larger than a normal fingertip touch.)

It will be recognized that distinguishing a palm contact from fingertipor thumb contacts can be especially difficult because the patch radiiresulting from a palm contact for people with small hands may approachthe patch radii caused by thumb or fingertip contacts for people withlarge hands. These types of contacts may also be distinguished inaccordance with the invention using the patch minor radius parameter.Referring to FIG. 4, illustrative empirical data is shown thatillustrates the distinction between palm contacts 400 and other contacts405 (e.g., fingertips and thumbs) based on patch minor radii. It hasbeen found that patch signal density values tend to be low for hoveringcontacts of any size, and saturate at a level independent of object sizeas the object presses firmly onto the touch-surface. Thus, the palmversus other object decision threshold 410 may be reduced for contactswith lower signal density because hovering or lightly touching fingersproduce lower minor radii than firmly touching fingers, whereas palmstend to produce large minor radii even when hovering. Accordingly,decision threshold 410 may be represented by a straight curve with asmall positive slope. While the exact values for patch contacts willvary as noted above, it is clear from FIG. 4 that threshold 410 may bemade to distinguish palm contacts from other contacts. Using thisapproach, there is virtually no risk that a hovering palm (a contactthat typically produces a patch signal density value similar to that ofa touching finger) will mistakenly be interpreted as a cursor move orbutton activation (e.g., a “click” event).

Ear and earlobe contacts can generate patches that are roughly the samesize as those generated by fingers and thumbs. It has been found,however, that the creases, ridges, and generally rough topography of theear do produce proximity images unique from fingers and thumbs, at leastif the imaging sensor (i.e., touch-surface) covers a significant portionof the ear (i.e. not just the fleshy lobule). The irregularity measuredescribed above is one way to characterize contact roughness (see EQ.5). This permits a robust means to discriminate between contacts due toears and earlobes from contacts due to fingers, thumbs, cheeks, thighsand palms. It has been found that the defined irregularity measure tendsto give values between 1.0 to 2.0 for ear and earlobe contacts whileregular (e.g., smooth) contacts attributable to fingers, thumbs, palmsand cheeks give values less than about 1.0. Referring to FIG. 5,illustrative empirical data is shown that illustrates the distinctionbetween ear contacts 500 and other contacts 505 (e.g., fingertips,thumbs and cheeks) based on the above defined irregularity measure. Inone embodiment, threshold 510 comprises a linear step-like or splinestructure with a first level at an irregularity measure of betweenapproximately 1.0 to 1.2 and a second level at approximately between 1.1and 1.2. In another embodiment, a single linear function having apositive slope may be used. In yet another embodiment, higher levelfunctions may be used to segregate the various contact types. As notedabove, while the exact values for patch contacts may vary from thoseshown in FIG. 5, it is clear that most rough object contacts may bedistinguished from most smooth or regular object contacts using thedefined irregularity measure—where the exact nature or form of adecision threshold (e.g., threshold 510) is dependent upon the preciseimplementation, operational goals and capabilities of the targetmulti-touch device.

In one embodiment, successive proximity images (aka “frames”) are usedto track objects as they move across a touch-surface. For example, as anobject is moved across a touch-surface, its associated patch(es) may becorrelated through overlap calculations. That is, patches identified insuccessive images that overlap in a specified number of pixels (orfraction of patch pixels) may be interpreted to be caused by the sameobject. In such an embodiments, the maximum patch minor radius over thelife of the tracked contact may be compared to the thresholds discussedabove (e.g., thresholds 310 in FIG. 3, 410 in FIGS. 4 and 510 in FIG.5). This approach ensures that a, for example, palm contact does notlose its palm identity should its minor radius temporarily fall belowthe decision threshold (e.g., 410). It is further noted that if adecision threshold is not a constant value (e.g., 310) but rather somecurve (e.g., 410 and 510), it may be advantageous to apply adensity-correction to the instantaneous minor radius prior to themaximum minor radius accumulation operation described here.

When taking a multi-touch device in and out of a pocket, or otherwisegenerally handling it, users should have the freedom to clasp their handaround it without producing spurious input. Such finger-clasps can bedetected via any one of the following criteria:

-   -   Identification (via block 125 in FIG. 1) of five, six or more        distinct surface contacts. (For a touch-surface the size of a        deck of cards, this many fingertips won't normally fit on the        surface, but since the phalange joints of each flattened finger        may get segmented into more than one contact patch, two or three        flattened fingers may generate five or more contact patches.)    -   Two, three or more contact patches are identified and the major        radius of at least two exceed approximately 15 millimeters to 18        millimeters. Since cheeks and other large body parts normally        produce just one patch with large major radius, the requirement        for two or three large patches prevents this test from        triggering on a cheek, leg or chest. Also, the requirement for        multiple large major radii prevents this test from triggering on        a couple fingertips accompanied by a long thumb laid flat        against the surface.

In another embodiment of the invention, multi-touch processingmethodology may include far-field processing. As used herein, far-fieldprocessing refers to the detection and processing associated with bodies(e.g., fingers, palms, cheeks, ears, thighs, . . . ) that are close to(e.g., less than one millimeter to more than a centimeter) but not incontact with the touch-surface. The ability to detect far-field objectsmay be beneficial in touch-surface devices that, during normal use, arebrought into close proximity to a user. One example of such a device isa telephone that includes touch-surface for user input (e.g., dialing).

Referring to FIG. 6, in one embodiment initial far-field processing maybe performed after proximity image data is acquired. That is, afteroperations in accordance with block 105 in FIG. 1. If the far-fieldmeasurement is designed to remain negative in the absence of any objectnear the touch-surface, and only become positive in the presence of alarge object, a first step subtracts a small noise factor from theinitially acquired proximity image to create a negative-backgroundfar-field image (block 600):Negative Far-Field Image=[PROX]−(Noise Factor)  EQ. 9In one embodiment, the noise factor may be set to between approximately1 and 2 standard deviations of the average noise measured or expectedover the entire image. This will cause most pixels in the resultingnegative far-field image to be slightly negative rather than neutral inthe absence of any touch-surface contact. As noted in FIG. 6, thenegative far-field image resulting from operations in accordance withblock 600 is denoted [NFAR].

Next, each pixel in the [NFAR] image is saturated to the highest levelexpected from an object hovering a few millimeters from thetouch-surface (block 605). In one embodiment, the resulting far-fieldsaturation image (denoted [SFAR] in FIG. 6) is generated as shown inTable 3.

TABLE 3 Illustrative Far-Field Saturation Operations For each pixel inthe initial far-field image: If [NFAR] > (Far-Field Saturation Limit)   [SFAR] = (Far-Field Saturation Limit) Else [SFAR] = [NFAR]

Since a goal of far-field operations is to be sensitive to large numbersof pixels only slightly activated (e.g., having small positive values),without being overwhelmed by a few strongly active pixels (e.g., havinglarge positive values), the saturation limit value should be less thanthe peak pixel value from fingers or thumbs hovering withinapproximately 1 to 2 millimeters of the touch-surface, but not so low asto cause the resulting [SFAR] image to lose to much information content.While the precise far-field saturation limit value will vary fromimplementation to implementation (due to differences in sensor elementtechnology and associated circuitry), it has been determined empiricallythat a suitable value will generally lie between +3 standard deviationsand +6 standard deviations of noise associated with the initialfar-field image. (Again, this noise may be on a per-pixel, or wholeimage basis.)

If the initial proximity image [PROX] contains a significant amount ofnoise, it may be beneficial to filter the [SFAR] image (block 610). Inone embodiment, a finite impulse response filter technique may be usedwherein two or more consecutive [SFAR] images are averaged together. Inanother embodiment, an infinite impulse response filter technique may beused to generate a smoothed image. It will be recognized that aninfinite impulse response filter generates a weighted running average(or auto-regressive) image. In one embodiment, for example, an infiniteimpulse response filter combines the current far-field saturated image(e.g., [SFAR]new) with the immediately prior far-field saturated image(e.g., [SFAR]prior) in a one-third to two-thirds ratio. As noted in FIG.6, a filtered far-field saturated image generated in accordance withblock 610 is denoted [FARF].

Following image segmentation operations in accordance with block 125(see FIG. 1), a weighted average of non-linearly scaled background pixelvalues may be used to generate a scalar far-field indicator value(FAR-FIELD) in accordance with the invention as follows:

$\begin{matrix}{{{FAR}\text{-}{FIELD}} = \frac{\sum\limits_{{{background}\mspace{14mu} i},j}{{{ENeg}( \lbrack{FARF}\rbrack_{{\lbrack i\rbrack}{\lbrack j\rbrack}} )} \times \lbrack {LOC}_{{\lbrack i\rbrack}{\lbrack j\rbrack}} \rbrack}}{\sum\limits_{{{background}\mspace{14mu} i},j}\lbrack {LOC}_{{\lbrack i\rbrack}{\lbrack j\rbrack}} \rbrack}} & {{EQ}.\mspace{14mu} 10}\end{matrix}$where the ENeg( ) function non-linearly amplifies pixel values below athreshold (e.g., zero) and [LOC] represents a pixel weighting mechanism.As indicated in EQ. 10, only proximity image background pixelscontribute to the computed FAR-FIELD value. That is, pixels identifiedas belonging to a patch during image segmentation operations areexcluded during far-field measurement operations.

In one embodiment, the ENeg( ) function disproportionately emphasizesthe contributions from background pixels as follows:

$\begin{matrix}{{{ENeg}( {{pixel}\mspace{14mu}{value}} )} = \{ \begin{matrix}{{pixel}\mspace{14mu}{value}} & {for} & {0 \leq {{pixel}\mspace{14mu}{value}} \leq B} \\{2 \times {pixel}\mspace{14mu}{value}} & {for} & {( {{- B} \div 2} ) \leq {{pixel}\mspace{14mu}{value}} < 0} \\{B + ( {3 \times {pixel}\mspace{14mu}{value}} )} & {for} & {{{pixel}\mspace{14mu}{value}} < ( {{- B} \div 2} )}\end{matrix} } & {{EQ}.\mspace{14mu} 11}\end{matrix}$where B represents a far-field saturation limit value. Empiricallydetermined, B is chosen to permit a small number of negative pixels tocancel out a finger or thumb-sized patch of positive pixels. In thisway, only a nearly full coverage cheek-sized patch of positive pixels,plus a small remainder of neutral/background pixels, can produce astrongly positive far-field measurement.

While not necessary, disproportionately emphasizing the contributionsfrom background pixels in accordance with EQs. 10 and 11 permits theFAR-FIELD measurement to be more selective for bodies large enough topositively affect most of a touch-surface's pixel (e.g., cheeks andlegs), while not being overly influenced by medium-sized objects (e.g.,hovering thumbs). For example, if a hovering thumb causes half of atouch-surface's sensor elements to have a slightly above-backgroundpixel value, disproportionately emphasizing the half that remain belowbackground will keep the measured FAR-FIELD value below zero—indicatingno large object is “near” the touch-surface (e.g., within 1 to 3centimeters). In another embodiment, background pixels may be linearlycombined (e.g., summed).

As noted above, [LOC] represents a pixel weighting mechanism. Ingeneral, there is one value in [LOC] for each pixel present in thetouch-surface. If it is desired to consider all touch-surface pixelsequally, each value in the [LOC] image may be set to 1.0 (or somesimilar constant value). For hand-held form-factors selectivity forlarge bodies may be improved, however, by lowering the weights near thebottom and side edges (for example, to values between 0.1 and 0.9).Doing this can lessen false-positive contributions from a hand whosefingers wrap around the device during (clasping) operations. In mobilephone form-factors, to retain sensitivity to ear and cheek far-fields,the weights along the top edge (where thumbs and fingers are less likelyto hover or wrap) may be kept at full strength.

Returning now to FIG. 1 at block 140, when far-field measurements aretaken into account during contact discrimination operations, a FAR-FIELDvalue greater than a specified threshold (e.g., zero) indicates a large“near by” object has been detected. As previously noted, thisinformation may be used to transition the touch-surface device into aspecified mode (e.g., on, off or low power). In addition, far-fieldmeasurements may be combined with other measurements (e.g., theirregularity measure) to provide improved ear detection. For example,when a touch-surface is partly against an ear and also hovering acentimeter or two from a cheek, a weak ear pixel patch may be segmentedin accordance with block 125 at the top of the screen. Meanwhile, themiddle and bottom of the touch-surface would only be affected by thecheek's far-field. Even if the FAR-FIELD measurement as taken outsidethe ear patch is not strong enough to exceed the specified far-fieldthreshold on its own, the FAR-FIELD value can be added or otherwisecombined with (weak) patch density and irregularity measure indicatorssuch that the sum or combination surpasses an ear detection threshold.

In addition, one or more proximity sensors may be positioned above thetouch-surface's top edge or around, for example, a telephone's receiveropening. Illustrative proximity sensors of this type include, but arenot limited to, active infrared-reflectance sensors andcapacitance-sensitive electrode strips. In a mobile telephoneform-factor, when the device is held such that the receiver is centeredon the ear canal, ear ridges may trigger the proximity sensor. Meanwhilethe earlobe may cause a small pixel patch in the top portion of thetouch-surface. Discrimination operations in accordance with block 140could decide that when a pixel patch at the top of the touch-surface isaccompanied by any significant receiver proximity trigger, the pixelpatch must be an ear, not a finger. In another embodiment, the sameconditions but with a significant FAR-FIELD value for the lower portionof the touch-surface (indicating a hovering cheek) may be used totrigger detection of an ear at the top of the touch-surface. Generallyspeaking, one or more of signal density (see EQs. 7 and 8), patchirregularity (see EQ. 5), FAR-FIELD measurement (see EQ. 10) andproximity sensor input may be combined (e.g., a weighted average) sothat ear detection can trigger when multiple indicators are weaklyactive, or just one indicator is strongly active. Finally, it is notedthat contact discrimination parameters such as a patches' centroid,minor axis radius, patch irregularity (EQ. 5), patch signal density(EQs. 7 and 8), far-field (EQ. 10) and proximity sensor input (ifavailable) may be (low-pass) filtered to help counteract their oftensporadic nature. This may be particularly beneficial if the filtersemploy adaptive time constants that rise quickly in response to risinginput values, but decay more slowing when input values drop and/or aremissing.

Referring to FIG. 7, a touch-surface device 700 of the type describedherein is shown in block diagram form. As used herein, a touch-surfacedevice is any device that receives user input from a multi-touch capabletouch-surface component (i.e., an input unit that provides user input inthe form of a proximity image). Illustrative touch-surface devicesinclude, but are not limited, to tablet computer system, notebookcomputer systems, portable music and video display devices, personaldigital assistants and mobile telephones.

As illustrated, touch-surface element 705 includes sensor elements andnecessary drive and signal acquisition and detection circuitry. Memory710 may be used to retain acquired proximity image information (e.g.,[PROX] image data) and by processor 715 for computed image information(e.g., patch characterization parameters). Processor 715 represents acomputational unit or programmable control device that is capable ofusing the information generated by touch-surface element 705 todetermine various metrics in accordance with FIG. 1. In addition,external component 720 represents an entity that uses the generatedinformation. In the illustrative embodiment, external component 720 mayobtain information from processor 715 or directly from memory 710. Forexample, processor 715 could maintain a data structure in memory 710 toretain indication of, for example, large body contact status, large bodyfar-field status, irregular object indication status, proximity sensorstatus (if utilized), flat finger clasp status and normal finger touchstatus. In one embodiment, each status may be indicated by a singleBoolean value (i.e., a flag).

Various changes in the materials, components, circuit elements, as wellas in the details of the illustrated operational methods are possiblewithout departing from the scope of the following claims. It will berecognized, for example, that not all steps identified in FIG. 1 need beperformed while others may be combined and still others divided intomore refined steps. By way of example, in one embodiment patchperipheral pixel noise is not suppressed (see block 130). In anotherembodiment, patch peripheral pixel noise suppression is employed but nopatch irregularity measure is made (see block 120). In still anotherembodiment, both patch peripheral pixel noise suppression and patchirregularity measures are determined and used. For embodiments that donot employ peripheral patch pixel noise reduction techniques, patchparameterization operations in accordance with block 135 use the [CNST]image and not the [CNST′] image as discussed above (see Table 2). Inaddition, patch parameterization operations in accordance with block 135do not need to rely on statistical ellipse fitting. They could insteadsum patch perimeter pixels and compare the obtained value to all patchpixels or attempt polygonal fits. Further, calibration operations (seeTables 1 and 2) may be delayed until, or made part of, imagesegmentation operations (block 125). In addition, it may be beneficialfor purposes of image segmentation to mark pixels that are at, or havebeen set to, the background level (e.g., during operations in accordancewith block 110). It is also noted that because the criteria foridentifying a finger clasp are orthogonal to large body contactdetection, flat finger clasps may be used as a distinct gesturecommanding a particular operation like locking the screen, going tosleep, or terminating a telephone call.

With respect to illustrative touch-surface device 700, touch-surfaceelement 705 may incorporate memory (e.g., 710) and/or processor (e.g.,715) functions. In addition, external component 720 may represent ahardware element (e.g., a host processor) or a software element (e.g., adriver utility).

Finally, acts in accordance with FIGS. 1, 2 and 6 may be performed by aprogrammable control device executing instructions organized into one ormore program modules. A programmable control device may be a singlecomputer processor, a special purpose processor (e.g., a digital signalprocessor, “DSP”), a plurality of processors coupled by a communicationslink or a custom designed state machine. Custom designed state machinesmay be embodied in a hardware device such as an integrated circuitincluding, but not limited to, application specific integrated circuits(“ASICs”) or field programmable gate array (“FPGAs”). Storage devicessuitable for tangibly embodying program instructions include, but arenot limited to: magnetic disks (fixed, floppy, and removable) and tape;optical media such as CD-ROMs and digital video disks (“DVDs”); andsemiconductor memory devices such as Electrically Programmable Read-OnlyMemory (“EPROM”), Electrically Erasable Programmable Read-Only Memory(“EEPROM”), Programmable Gate Arrays and flash devices.

The invention claimed is:
 1. A method to process a proximity image,comprising: receiving the proximity image; generating a dispersionimage, wherein the dispersion image is based on a high-pass filteredversion of the proximity image; determining an irregularity measurevalue for the dispersion image, the irregularity measure valueindicating a degree of irregularity in a shape of the dispersion image;and controlling an operation of a touch-surface device when theirregularity measure value is above a specified threshold.
 2. The methodof claim 1, wherein the specified threshold comprises a constantfunction.
 3. The method of claim 1, wherein the specified thresholdcomprises a linear function.
 4. The method of claim 1, wherein thespecified threshold comprises a non-linear function.
 5. The method ofclaim 1, wherein the specified threshold comprises two or more linearfunctions.
 6. The method of claim 1, wherein the operation of thetouch-surface device comprises causing the touch-surface device toobtain baseline touch-surface sensor values.
 7. The method of claim 1,wherein the operation of the touch-surface device comprises: utilizingmultiple sampling frequencies for operation of the touch-surface device;determining an irregularity measure value corresponding to each of themultiple sampling frequencies; ignoring a touch-surface sensor samplingfrequency during touch-surface device operations when a correspondingdetermined irregularity measure value is greater than the specifiedthreshold.
 8. The method of claim 1, wherein the operation of thetouch-surface device comprises causing the touch-surface device to enterinto a low-power state.
 9. The method of claim 1, wherein the operationof the touch-surface device comprises causing the touch-surface deviceto exit a low-power state.
 10. The method of claim 1, wherein thetouch-surface device comprises one of the devices selected from thegroup of tablet computer system, notebook computer system, portablemusic player, portable video player, personal digital assistant andmobile telephone.
 11. A non-transitory program storage device, readableby a programmable control device, comprising instructions stored thereonfor causing the programmable control device to perform the method ofclaim
 1. 12. A portable electronic device, comprising: a touch inputtouch-surface component of the touch-surface device; means for receivingthe proximity image from the touch input touch-surface component; andprocessing means for performing the acts of smoothing, generating,determining and controlling in accordance with claim
 1. 13. The devicein accordance with claim 12, wherein the processing means is operativefor: determining a far field image from the proximity image; andcontrolling operation of the mobile telephone based on both thedetermined irregularity measure value and the far field image.
 14. Thedevice in accordance with claim 12, wherein the portable electronicdevice comprises is selected from the group consisting of a tabletcomputer system, a hand-held computer system, a portable music playersystem, a portable video player system and a mobile telephone.
 15. Amethod to identify an irregular input source to a touch-surface device,comprising: obtaining a proximity image; generating a dispersion imagefrom the proximity image; segmenting the proximity image to identify aplurality of patches; determining an irregularity measure value for eachof the plurality of patches based on the dispersion image, theirregularity measure value indicating a degree of irregularity in ashape of each of the plurality of patches; identifying one of theplurality of patches as being associated with an irregular object whenthe irregularity measure value of the one patch is above a specifiedthreshold; controlling an operation of the touch-surface device based onthe identification of the irregular object; wherein the dispersion imagecomprises a high pass filtered version of the proximity image; andwherein the act of determining an irregularity measure value for each ofthe plurality of patches comprises at least determining a total energyvalue for each of the plurality of patches.
 16. The method of claim 15,wherein the plurality of patches have patch pixel values and the methodfurther comprising: compensating the proximity image for backgroundnoise; filtering the proximity image; and reducing or down-scalingperipheral patch pixel values for one or more of the plurality ofpatches based on the compensated proximity image and the filteredproximity image.
 17. The method of claim 1 wherein the irregularitymeasure indicates a general roughness or non-roundness in the dispersionimage.
 18. A method to process a proximity image to discriminate betweenear and earlobe contacts as distinguished from other contacts in amutual capacitance touch surface mobile telephone device, comprising:obtaining the proximity image based on a disturbance in an electricfield of a two dimensional array of sensing elements of the mutualcapacitance touch surface mobile telephone device; generating adispersion image based on a high-pass filtered version of the proximityimage; segmenting the proximity image to identify a plurality ofpatches; determining an irregularity measure value for each of theplurality of patches based on the identified patches and the dispersionimage, wherein the irregularity measure value indicates a degree ofirregularity in a shape of the dispersion image associated with the earor earlobe positioned near or in contact with the mutual capacitancetouch surface mobile telephone device; identifying one of the pluralityof patches as being associated with the ear or earlobe contact or nearcontact if the irregularity measure value of the one patch is above aspecified threshold; and controlling the operation of the mutualcapacitance touch surface mobile telephone device based on theidentification of the ear or earlobe contact.
 19. A method to process aproximity image, comprising: segmenting the proximity image to identifya plurality of patches; generating a dispersion image, the dispersionimage based on a high-pass filtered version of the plurality of patchesof the proximity image; determining an irregularity measure value foreach of the plurality of patches of the dispersion image, theirregularity measure value indicating a degree of irregularity in ashape of each of the plurality of patches of the dispersion image;identifying at least one of the plurality of patches as being associatedwith an irregular object when the irregularity measure value of the onepatch is above a specified threshold; and controlling an operation of atouch-surface device if the irregularity measure value is above aspecified threshold.
 20. The method of claim 19, wherein the operationof the touch-surface device comprises: utilizing multiple samplingfrequencies for operation of the touch-surface device; determining theirregularity measure value corresponding to each of the multiplesampling frequencies; ignoring a touch-surface sensor sampling frequencyduring touch-surface device operations when a corresponding determinedirregularity measure value is greater than the specified threshold.